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Abstract 

This paper provides a suggested teaching model enabling students to conduct 
extensive, hands-on survey research as the basis of part or all of a political 
science class. The model emphasizes active student learning and development 
of applied skills. The components of this model can be modified for utilization in a 
broad array of undergraduate political science courses, such as political science 
methods, campaigns and elections, public opinion, and public policy. 

The focus of the paper is upon aspects of the model which albw for student 
training; questionnaire creation; development of interviewing/oral communication 
skills; data collection, entry and analysis; and issues of statistical interpretation. 
Special attention will also be given to the issue of establishing evaluation 
mechanisms of student performance. 
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Introduction 

The suggested model presented below developed out of an initial social political 
research project focusing on a county-wide (Westchester County, New York) 
telephone survey regarding social and political issues. Students were recruited 
from various disciplines, paid a stipend for their participation and were actively 
involved in all phases of the research process. Students collected data via 
telephone surveys after receiving training on basic interviewing and 
communication skills. Students also participated in questionarre construction 
and data collection, coding, entry and analysis. 

After completion of this project, the involved faculty members realized the 
potentially significant educational benefits students can receive through 
involvement in similar research exercises applied in actual courses, such as 
social science methods or upper division political science classes. Among the 
most important of these benefits is providing students with a “real life" experience 
whereby they can develop useful skills and a better understanding of and 
appreciation for the research process. Additionally, the faculty members casually 
observed that participating students were highly engaged in the overall process 
and found the work to be both interesting and challenging. We believe such 
student sentiments, if channeled properly, can lead to higher degrees of student 
satisfaction with a particular course, the political science discipline, and 
contribute to a more positive overall college experience. 

This paper (and it’s corresponding conference presentation) is intended to 
share the author’s experiences and perspectives and to propose a possible 
model for utilization of some or all of the components of the polling/survey 
process into an upper division undergraduate or graduate political science 
course. The following analysis includes a brief mention of some key literature, a 
description of the primary components of the employed polling method, and 
suggestions for establishing student performance evaluation criteria. 



Some Relevant Literature 

Works from the education literature, and to some degree the psychology 
literature, examine topics regarding the nature of the educational process and 
student learning. The following brief review essentially draws from some of this 
literature while focusing upon the two primary educational benefits of our 
proposed model; 1) engaging students in active learning while creating a positive 
educational experience, and 2) developing “real life” skills through critical 
thinking. 

-Engaging students in iearning whiie creating a positive educationai experience- 
Ernest Boyer noted in his widely renowned Carneige Foundation Report on the 
advancement of teaching noted, among many other things, that students enter 
college with very high expectations. They, along with their parents, expect 
“something special” from the college experience (Boyer, 1987). But, key 




2 



questions quickly jump to mind; how may the college experience come to be 
seen as special?; what factors lead to a positive experience for students? 

Much of the research in this area focuses upon the importance of 
integrating students into the academic environment and intellectual process (See 
Christie and Dinhome, 1991; Johnson, et.al., 1995; Tinto, 1993). Faculty play a 
central role in this integration process through engaging students in a challenging 
and interesting academic program. For example, Smith (1977) argues that 
academic involvement both in and out of the classroom is strongly related to 
student satisfaction and success. Smith continues that faculty encouragement 
along with other factors such as peer interaction is essential to student 
development. Not surprisingly, developing good academic exercises is related to 
student integration, satisfaction and even retention. 

Pascarella and Terenzini (1991) also note the centrality of the role of 
faculty in student development and retention. Their research indicates that 
students believe faculty members should help them achieve their goals. If the 
faculty exhibit skill in teaching, good rapport, structured classrooms, good 
interaction skills, and the willingness to give ongoing feedback to students, then 
there is a greater chance that students will become engaged in the intellectual 
process and the larger academic community. In sum, good pedagogical skills 
and clear organization were key factors, perceived by students, for achieving 
success. 

The model outlined below offers several opportunities for integrating 
students into a challenging intellectual process and for enhancing pedagogy. 
The possibility of student integration is high because the survey process requires 
a significant degree of instructor contact and peer interaction inside and outside 
of the classroom, particularly during the actual calling/interviewing of participants. 
We found that both students and participating faculty quickly develop a positive 
rapport with one another during the entire process. Pedagogy is also enhanced 
because the process allows for a broad array of techniques that reinforce student 
learning. 

-Developing “real life” skills- 

The teaching model explained in this paper provides the opportunity for 
experimentation beyond a traditional mode of instruction and learning by lecture 
to an environment where students can develop critical thinking skills by 
experiencing the research process first hand. In our model, students actually 
create the survey, do the interviews, collect and enter data, and analyze the 
results which cumulatively provides a rich and experiential based context to apply 
knowledge. As O'Rorke has argued, linking critical thinking exercises to 
contextual learning is key to overcoming the argument that only courses or 
exercises which teach "practical skills" are worthwhile to students (O'Rorke: 15) 
Moreover, O'Rorke notes, 

"Students tend to be somewhat short sighted in their perspective on the 
value of college courses. It is commonplace for students to evaluate the 
context of courses based on their perspective of the course's relevance to 
their future employment opportunities. Those courses, and components of 
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courses, which they believe will directly enhance their employment 
opportunities are taken more seriously and engaged in more 
conscientiously. Those elements of the course, and curriculum for that 
matter, for which students cannot discern direct impact on their future 
prospects are treated more casually and with less enthusiasm. Students 
are conditioned to think that "practical," technical skills are most important 
to future success and thus, undervalue the worth of exercises and projects 
which attempt to develop higher level thinking skills over the long term." 
(0’Rorke:14) 

Thus, it is a major challenge to the college instructor to create mechanisms which 
allow students to learn while linking this learning process to the context of 
students' experiences. Our model provides such a linkage because substantive 
knowledge is wedded to the students' experiences in conducting an actual 
survey. 

Our model provides a linkage between knowledge and experience as well 
as allowing for the development of critical thinking skills. Young (1980) has 
recited several learning activities that comprise critical thinking, such as; the 
ability to define a problem, the ability to select pertinent information for a solution, 
the ability to recognize assumptions, and the ability to draw valid inferences and 
conclusions. As will be demonstrated, the development of these critical thinking 
shills is an inherent part of the model described below. 

Key Components of a Suggested Teaching Model 

One of the first things that should be emphasized with students is the fact that 
surveys/polls are commonplace in academic research across many disciplines as 
well as in the pubic and private sectors. Many examples of the different types of 
polls should be discussed (see The Survey Kit series published by Sage 
Publications, 1995). However, students should also be required to think about 
some of the problems and limitations of surveys, particularly political opinion 
polls. For instance, public opinion can change rapidly thereby limiting the 
applicability of polling results over time. A full discussion of survey research 
limitations is absolutely necessary. 

Next, since polling involves several interrelated steps, each component of 
the polling process should be clearly broken-down and explained to students 
prior to the undertaking of an actual survey or project. A suggested breakdown 
could be as follows; 

1 . determining focus and objectives of the survey, 

2. developing survey questions (questionnaire construction), 

3. student training in conducting survey research, 

4. understanding key statistical issues, 

5. data collection, entry and analysis, 

6. evaluating student performance. 

The following analysis focuses upon these steps. 
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-Determining objectives and questionnaire creation- 

Objectives of the survey need to be agreed upon before the construction of a 
survey instrument or questionnaire. An instructor can pre-determine an objective 
given what he/she thinks would be appropriate, perhaps focusing upon a local 
election or compelling policy issue. Alternatively, the instructor can allow 
students to make suggestions and then arrive at some consensus. This latter 
approach would allow for greater student involvement and perhaps wed them to 
the project early on in the process. In any event, an instructor has to make a 
professional assessment of how much involvement he/she wants in the setting of 
the objectives. Once an objective has been clearly specified, the next step is to 
develop the appropriate survey instrument/tool. 

In the development of a sound survey tool, one does not have to reinvent 
the wheel. In many instances, there are a number of good survey tools currently 
in use whereby one may pick and choose items that are relevant to their own 
interests. In this process one must be careful of copyright restrictions since some 
questions may be in the public domain while others may not. Obviously, it is 
always wise to check with the author for permission to use his/her items. 

Regardless of the source of the items there are some common errors that 
must be avoided. What follows is a brief, but not exhaustive, list of things to be 
avoided. 

1 . Survey items must be worded in an unambiguous manner. One should 
avoid using words such as may, might or possibly. “Do you agree or 
disagree with the following statement. Inflation may become a problem 
in the coming year.” A more specific question would be, " Do you think 
that inflation is a problem?" 

2. Questions should be clearly stated avoiding the use of technical 
language. “Do you agree or disagree with the following statement. 
Greater state funding for prisons will reduce the subculture of 
violence amongst the prison population.” Always assume your 
audience is intelligent, but uninformed about a given topic. 

3. Questions should be objective. “Should we spend money to educate 
America’s children?” This is a bad question because there are 
probably very few people who would answer in the negative. This is a 
“loaded” question, and such questions should be avoided. Such a 
question would have to be specified. 

4. Avoid the use of “Double Barreled” questions. “Should government 
spend more money on health care and less money on drug 
rehabilitation programs?” This question does not allow for people with 
two different views on the two issues linked in the question. 

5. Avoid the use of negative questions. “New York State should not give 
benefits to new immigrants.” This type of question is often misheard 
during a telephone survey. 

There are some additional characteristics good survey questions should 
posses. 

1 . Shorter questions are better than longer questions. 

2. Questions should be relevant to the population under study. 
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3. Questions should be worded in such a way that respondents are willing 
to answer. 

Anyone interested in more details on question construction should see E. Babbie 
The Basics of Social Research Belmont: Wadsworth Publishing Co., 1999: pp. 
126-138. 

It is common practice among researchers to undertake a “pilot study” prior 
to doing the actual research survey. A pilot study administers the survey 
instrument to a small number of respondents. After this is completed the 
researchers can analyze the pilot experience looking for problematic questions. 
These questions may be reworked or eliminated. 

-Student training and skill development- 

Students who participated in our initial project were selected based on criteria 
that included a relevant major (social sciences) and demonstration of strong oral 
communication skills. Actual classes of certain upper-level major courses, such 
as social science methods, were also incorporated into the phone interviewing 
portion of this project. Even though the initial project was not conducted as an 
actual course, the requisite training of students would still be identical. 

Students were required to participate in a training session on proper 
phone etiquette and interviewing skills. The training session focused on teaching 
students helpful strategies to enhance the likelihood of a successful phone 
interview. These strategies included the use of a standardized opening 
statement for each interviewer to read and rehearse. The prepared statement 
was used to eliminate any hesitation by the interviewer in the vital first few 
seconds of a call. The rehearsal of this statement also allowed for the 
interviewer to reduce the number of verbal errors while making the student more 
comfortable with the survey process. 

A key phone interviewing strategy taught involved the tone of voice used 
and the building of a rapport with a possible participant. Students were 
encouraged to use a friendly, non-threatening voice when attempting a phone 
survey interview. The interviewers were also taught to emphasize that they were 
students from the College, since we found that people are more inclined to help a 
student, especially from an area school. The students were also instructed to 
establish an immediate rapport with the possible participant by stating their first 
name in their opening sentence, to avoid an overly formal tone. 

Another strategy utilized in this project was to include reassuring 
comments during the interview. The students were told to be honest with the 
possible participant from the beginning, for example, by explaining to participants 
that it would take approximately 10 minutes to complete the survey (this time 
would be considerably shorter if the survey is only a few questions). The 
interviewers were also encouraged to make it clear to the possible participant 
that their responses were confidential, the information would not be made 
available for sales purposes or to telemarketers, and that their phone number 
was selected by a random sampling of phone numbers and would not be reused. 
By addressing these issues in the beginning of the interview, the student was 
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attempting to eliminate any apprehension that the possible subject might have 
had about participating and to increase the overall number of completed surveys. 

Some students became very adept at convincing phone respondents to 
participate, especially those individuals who were initially hesitant to participate. 
For example, students emphasized that the interview was for educational 
purposes or students frequently asked to call a respondent back if they were 
busy at the time of the initial call (many participants would Just take the time at 
the moment instead of anticipating a call back). Other students emphasized the 
political and practical importance of the poll whereby public opinion is measured 
and reported which may, in turn, impact on elected officials and public policy. On 
this last point, it is imperative to prevent students from debating the merits of 
polling with a prospective participant. In such an event, thank the caller and 
move on to another call. 

Students were also trained on additional techniques to help maximize the 
chances of successfully completing an interview. Interviewers were taught to 
maintain a participant's interest in the survey by reassuring that they were 
nearing the end of the survey after a couple of minutes interviewing. The 
interviewers were also reminded to read each question slowly and clearly and be 
willing to repeat each question as many times as necessary without indicating 
any annoyance. A good technique would be to tape record a student, play the 
recording back to them, and have them identify any unclear elements of their 
speech. 

The interviewers were cautioned to strictly refrain from interjecting their 
own opinions about the issues being discussed in the survey. Students were 
also directed not to explain or to provide additional information concerning any of 
the political questions or political candidates mentioned in the survey. This was 
done to eliminate any possible effects of the interviewer's opinions on the 
participant. 

The interviewers were also trained to be courteous and polite throughout 
the phone interview. They were told to thank the participant for their assistance 
after completing a survey. Students also thanked a person for their time, even if 
they did not want to participate. 

Faculty members encouraged students to adopt a style that suited them 
best and that was flexible enough to incorporate the above mentioned strategies. 
Overall, the average response rate was about the same for most students. 

-Understanding statistical issues- 

Students who participate in experiential learning revolving around the 
development and execution of a telephone survey can be expected to have 
statistical questions. This list of questions may include several of the following: "If 
we want to survey the city of X, rather than all of Y County, or State Z, how much 
smaller is the required sample size?" "What does it mean when you say the 
difference between two groups is significant?" "Is there a difference between 
significant differences and important differences?" and "How good is the sample 
we collected?" The answers to these questions will provide students with the 
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basic understanding of applied statistics that every social scientist should 
possess. 

Students are always surprised to learn that there is no difference in the 
sample size required to survey a large city, the county in which the city is located, 
a state or an entire nation. To begin to understand why this is true, consider a 
study in which the height of the participants is recorded. If you were told that a 
single subject was drawn from the participants and that the person was 67" tall, 
you might comment that the person was tall but would have no reason to 
question the result. However, if you were told that 100 participants were drawn 
and that their average height was 6"7", you would question the validity of this 
result. Why? Because your experience tells you that the average height of a 
group of 100 people will be closer to the true average height than the result 
reported. That is, the larger the group the more representative of reality you 
expect its results to be. Notice that the question and criticism of results arose 
without knowing if a town, city or state was being surveyed. Thus, it seems that it 
is the size of the sample that determines its chance of being representative of a 
population, and it seems that that size is not dependent on the size of the 
population being studied. The reason for the truth of this statement lies in the 
fact that a sample of any size is a quality sample if it is representative of the 
population from which it is drawn. Thus, when determining the sample size 
needed, one must only calculate the size needed to insure a representative 
sampling. 

When reviewing the results of an involved survey it is possible that a 
student might see a report statement similar to the following, "The difference of 
7.3% between the views of males and females on the need for better after-school 
programs was statistically significant." To appreciate this statement students 
must be informed that the assumption that underlies all the analyses is that there 
is, in truth, no difference between groups and any observed differences are due 
to the natural variability in samples. This natural variability produces a range of 
results, since each sample contains a different set of subjects. When samples 
large enough to be considered representative are taken some results, those 
close to the true population value, occur frequently. Others, those far away, occur 
less frequently. Students must understand that the above report statement 
implies that this observed difference is considered too large to be reasonably 
explained by natural sampling variation. That is, the observed difference 
happens so infrequently by chance that we are more comfortable attributing the 
result to real differences in the attitudes of males and females rather than to 
sampling variation under the original assumption that the attitudes of males and 
females are the same. 

Consider a situation in which the 14% difference between the 64% of the 
males and 78% of the females who stated they would vote for candidate A is 
significant. Is this difference important? Not to candidate A. During the 
classroom discussion of this point it is interesting to show students how 
dependent significance is on sample size. Using appropriate technology 
students can see that the 1.5% difference between 2.5% and 4.0% is not 
significant if these numbers are generated by a sample of size 200 (5/200 & 
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8/200), but is significant when generated by a sample of size 2000 (50/2000 & 
80/2000). These examples clearly illustrate that significance is a statistical 
concept, while importance is a practical matter. 

The final question that will be investigated is that of sample quality. One 
of the first things that students participating in survey work observe is. the number 
of no answers and refusals. They often ask if they should call back these 
numbers at a later time. They are usually surprised to hear the answer to this 
question is an unqualified YES. It is this short exchange which suggests the 
need for a discussion of randomization and sampling bias. Students should be 
informed of the procedures used to generate the telephone lists they are using to 
make their calls. This discussion should include an exercise in which students 
use random number tables or random number generator to create several lists of 
random subjects from a population with known characteristics. For example, a 
list of 100 names, of which 10 are identified as having green eyes, can be used 
as the population to be sampled. Students can be asked to draw 5 lists each 
containing ten random names. The concept of bias and sample variation can 
then be illustrated using these student generated lists. Finally, the potential bias 
due to non-response should be discussed. 

The classroom time spent on the questions and concepts reviewed above 
will provide the students with a solid grounding in the basic statistics of sampling 
and will provide them with a deeper appreciation of the survey process in which 
they have participated. 

-Data collection, entry, analysis- 

Data collection took place over a two-week period in the Spring of 1999. Paid 
student volunteers made phone calls to randomly chosen Westchester county 
residents weekday evenings between the hours of 4:00 p.m. and 9:00 p.m., and 
Saturday between the hours of 10:00 a.m. and 5:00 p.m. Students who were 
enrolled in various research methodology courses also volunteered to make calls 
at various times. The list of random Westchester county residents was provided 
by a professional service specializing in survey methodology. In all, 603 survey 
forms were completed within this timeframe. This number met the design goal of 
601 completed surveys and thus provided the ± 4% margin of error sought. 

Data entry was done shortly after completion the surveys. Each student 
volunteer entered data onto a SPSS spread sheet and saved that database in a 
file named with their initials. SPSS is a commonly available and widely used 
statistical program. Once all the data had been entered in this manner the files 
were merged into a master database. After a brief tutorial, students reported that 
data entry in SPSS was easy and could be done quickly. A feature of SPSS that 
weighted heavily in its favor was its Value Label feature that is easily understood 
and implemented. This feature allows the data entry team to create labels for 
common answers to a question, i.e. 1 = Democratic, 2 = Republican, 3 = 
Independent, etc. Then, at the time of data entry, the label rather than the literal 
response can be typed into the database. When viewing the data the researcher 
can request to see either the label or literal response. This proved to be a great 
time saver. 
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The primary purpose of this survey was to deveiop descriptive measures 
of Westchester county residents' attitudes and opinions toward a variety of 
poiiticai and sociai issues. Thus, the anaiysis of the resuits of this survey 
consisted primariiy of descriptive statisticai measures such as counts, 
percentages and modes. Counts and modes are easiiy caicuiated by SPSS 
using the Frequencies command that can be found in the Summarize submenu 
of the Statistics menu. The data anaiyst must simpiy inform the program of the 
name of the variabies for which counts and percentages are required, ciick OK 
and the program does the rest, if modes are required the operator must first 
check off mode in the statisticai options box that SPSS makes avaiiabie to the 
user and then proceed as above. CrossTabs are aiso easiiy obtained using 
SPSS. The command that generates crosstabs is aiso found in the Summarize 
submenu of the Statistics menu. This type of anaiysis, for exampie, can be used 
to find out how gender can predict voting preferences. Are maies more iikeiy than 
femaies to vote for Repubiican candidates? This type of anaiysis can be taken 
one step further by examining how two variabies, gender and education, 
influence voting preferences. How do the voting preferences of highiy educated 
femaies compare to the voting preferences of iower educated maies. The SPSS 
program aiiows for such an anaiysis by simpiy naming the variabies that are to 
be used in the division process and then clicking on OK. The actuai anaiyses that 
were performed were those requested by members of the research team with 
specific research questions they wished to have answered. The resuits of these 
anaiyses were summarized in the press reieases which are discussed eisewhere 
in this report. 

-Developing student evaluation mechanisms and assignments- 

The nature of this type of learning experience may not iend itseif weii to 
traditional modes of student evaluation, such as examinations. There are, 
however, several strategies professors may use to evaluate the performance of 
students who participate i n an experiential learning exercise. 

Journai: Students maintain a journal in which they record their 
experiences and reactions to the poiiing research process. The journal entries 
shouid focus on what they have learned. The professor may review the journai 
on a weekiy basis; or, the journal may be submitted at the end of the semester 
for evaluation. The basis of evaiuation shouid be on iearning outcomes. 

Term Paper: Students may be requested to submit a traditional term 
paper. There are several options that may be presented to students: a) focus on 
one element of the polling project and offer an extensive analysis of that element 
including a review of the literature, data presentation and analysis, and the 
drawing of conciusions. Suggestions for future research should also be included; 
b) students may be asked to create their own poiiing project. One semester 
wouid not afford them the time to carry out the poll; however, students may 
create hypothetical questions, and offer an evaluation based upon what tiey 
beiieve they would find if the poil was to be carried out (the project shouid inciude 
the use of “dummy tables”). 
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Role Play Written Assignments: Students may be put in the position of 
political consultant or analyst who has been hired to examine a specific area or 
question and produce a formal report. For example, a political candidate for 
office wants to know his strongest contingent of potential supporters and what is 
important to them. This can be given to the students at the beginning of the 
process whereby they can craft the survey around this final goal. 

Press Releases: Throughout the semester, as polling data is gathered, 
students may be requested to create a series of press releases which report on 
the key conclusions of the survey. This would give students the opportunity to 
differentiate between important and less important findings, develop critical 
thinking skills, and hone their writing skills. One may even decide to actually 
release the results to the media. 

More traditional evaluation techniques can still be employed, such as: 

Exams and Quizzes: The professor may develop a series of quizzes or 
examinations that focus on the polling research process. For example, utilizing 
the real data collected, and/or fictional data, students may be asked to interpret 
and analyze the data presented. 

An instructor may utilize any one or combination of approaches to 
evaluate students. 

SUMMARY 

All the faculty who were involved in the initial project which is the basis of the 
above model where struck with the high degree of student enthusiasm with the 
survey. We intuitively felt the need to share our professional experiences 
because we believe that the proposed model offers some real and innovative 
possibilities for improving student learning. 
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